Skip to content

feat(s101): native S-101 (S-100 Part 10a) chart loading#16

Merged
beetlebugorg merged 12 commits into
mainfrom
feat/s101-native
Jul 13, 2026
Merged

feat(s101): native S-101 (S-100 Part 10a) chart loading#16
beetlebugorg merged 12 commits into
mainfrom
feat/s101-native

Conversation

@beetlebugorg

Copy link
Copy Markdown
Owner

Reads native S-101 ENC charts directly, auto-detected from the .000 file, feeding the portrayal pipeline without the S-57→S-101 adapter (native data already speaks the S-101 vocabulary). The public API is unchanged — png, bake, and the C ABI accept a native S-101 chart transparently.

  • Reader + assembly (src/s101/dataset.zig, native.zig): reuses the ISO 8211 container; decodes the S-100 spatial + feature model (in-band FTCS/ATCS code tables → S-101 class/attribute names, ATTR complex-attribute tree). Maps the S-100 spatial records onto the existing S-57 vector model, so the whole geometry/masking/boolean/scene pipeline is reused; builds adapter.Adapted directly.
  • Update files (.001+): record-level insert/delete/modify by (RCNM, RCID), with per-association SAUI edits; names resolve per-file (each carries its own code tables).
  • Pick report: serves each native feature's S-101 class + full attribute tree (nested complex attributes), UTF-8 preserved.
  • Band scale: derives cscl from the DataCoverage display scale so charts bake at their correct band (an overview test cell dropped from >200s to ~1s).
  • fix(iso8211): tolerate nested parentheses in DDR format controls (UKHO cells use them; every UKHO chart previously failed to load).
  • Tooling: tile57 s101 inspector; docs.

Validation: SHOM France + UKHO GB production datasets (incl. update chains) and the IHO S-164 update conformance set — all load with 0 unresolved codes / 0 portrayal errors; S-57 rendering unchanged; every chart bakes ≤1.7s.

Add src/s101/dataset.zig: an ISO 8211-based reader for native S-101 .000
files. It reuses the shared iso8211 container and decodes the S-100 General
Feature Model — DSSI coordinate factors + record counts, the in-band code
tables (FTCS/ATCS/ITCS/ARCS mapping numeric codes to S-101 class/attribute
names), and the point/multipoint/curve/composite-curve/surface spatial
records plus feature records (FRID/FOID/ATTR/SPAS/FASC/MASK). ATTR decodes
the complex-attribute tree via the PAIX parent-position links.

detect() discriminates S-101 from S-57 on the DDR field schema (the S-101-only
FTCS code-table field). Add a 'tile57 s101' subcommand to inspect a dataset.

Validated on all 8 SHOM France test cells: parsed record counts match the
DSSI counts exactly and every feature code resolves to a class name; a real
S-57 cell is correctly not detected.
Add src/s101/native.zig. parseDataset(gpa, bytes) maps a native S-101 dataset
onto the two things the renderer consumes, bypassing the S-57->S-101 adapter:

  * a geometry-carrying s57.Cell shell — S-100 points -> VC nodes, curves ->
    VE edges (begin/end from PTAS + interior C2IL), multipoints -> VI sounding
    vectors; features resolve their SPAS (and, for surfaces, RIAS rings and
    composite-curve members) into FSPT-style edge/node refs with ORNT/USAG/MASK.
    Every existing geometry, masking, and boolean accessor then works unchanged.

  * []adapter.Adapted built directly — class name from the FTCS table, the CNode
    attribute tree reconstructed from ATTR's PAIX parent-position links (native
    S-101 already carries the complex attributes the adapter has to synthesize
    from flat S-57 attributes). A small class->objl / name->attr-code surrogate
    covers only the classes scene.zig special-cases (soundings, depth contours,
    dangers, coastline masking, lights).

Validated on all 8 SHOM France cells: features==adapted, vectors==edges+
soundingVecs, nodes==points; area/line/point/sounding geometry all resolve
through the s57.Cell accessors with bounds on the French coast.
…peline

Wire the native S-101 path into every .000 load site so the same API renders
S-101 and S-57 charts transparently:

  * portray: add portrayCellWithAdapted / portrayCellVariantsAdapted — the
    native entry points that portray a pre-built adapter.Adapted set, skipping
    adapter.adaptCell. portrayCellWith / portrayCellVariants now delegate to them.
  * chart.zig: parseAnyCell() detects S-101 vs S-57 from the file and routes to
    s101.native.parseDataset or s57.parseCellWithUpdates; the lazy loader, the
    query backend, and the bake worker use it + portrayVariantsAny.
  * render.zig (tile57 png/pdf): same detect-and-route for the live-context path.
  * s57.Cell gains a  flag; scene.zig's 'unknown feature -> QUESMRK1'
    fallback is gated on !cell.native (a native feature always has a valid S-101
    class, so a null portrayal stream means 'nothing to draw', not 'unknown').
  * native.zig: mark the shell native; exclude SOUNDG + geometry-less features
    from the adapted set (scene emits soundings directly).

A French S-101 cell now renders a full S-52 chart (depth areas, soundings,
contours, coastline, buoys/beacons, wrecks, dangers, text, routes); a known
S-57 cell renders unchanged.
Map the S-101 scaleMinimum attribute onto the S-57 SCAMIN code (133) so native
features are scale-culled by the existing S-52 gate — a low-zoom view of a
French cell drops from 11159 to 2093 draw ops, matching S-52 generalization
(small-scale soundings/marks hidden until zoomed in).

The 'tile57 s101' command now also runs the rules and reports ok/empty/ERROR
portrayal-stream counts by class: all 8 France cells portray with 0 errors
and 0 empty streams.
tile57 reads both native S-101 (S-100 Part 10a) and legacy S-57 charts, auto-
detected from the .000 file. Update intro + architecture pipeline to show the
two converging entry paths, document the 'tile57 s101' inspect command, and note
in limitations what native S-101 does not yet wire (update files; feature/info
association surfacing in queries).
parseWithUpdates merges a native S-101 base cell with its sequential update
files at the record level, by (RCNM, RCID): RUIN 1=insert, 2=delete, 3=modify
(S-100 Part 10a). A feature MODIFY applies the per-association SAUI edits
(2=drop the matching spatial association, 1=add) and replaces attributes/FASC
when the update carries them.

Because each file (base and every update) has its OWN in-band code tables,
class/attribute NAMES are resolved per file at decode time and stored on the
records; all kept bytes are duped into the dataset arena, so the per-file ISO
readers are transient (the Dataset no longer holds one). detect + parse are
unchanged for a base-only cell.

Wired through s101.native.parseDataset, chart.zig parseAnyCell, render.zig, and
the 'tile57 s101' inspector (all auto-apply the sibling .001.. chain).

Validated on the S-164 10100AA_X01SW conformance set (edition-1.0 base + updates
1-5): the merged result is byte-for-byte identical to an independent record-level
merge simulation (all 795 features, every class), and portrays with 0 errors.
Unit tests cover the rcid merge index and the SAUI modify path.
Update the limitations note: native S-101 charts apply their .001.. update chain
(record-level insert/delete/modify), validated on the S-164 conformance set.
Feature/info association surfacing in the query report remains the one gap.
The cursor-pick / feature-info report read S-57 f.objl + f.attrs, which for a
native S-101 cell are only surrogates — so a feature's real S-101 attributes
(including complex/information text like a French leading-line note) never
appeared, and any that did risked mangled encoding.

native.zig now builds, per feature, the S-101 class name + a JSON of the S-101
attribute tree (simple values + nested complex attributes) straight from the
CNode, and stores them on the s57.Cell shell (pick_class / pick_json). The JSON
encoder escapes only the JSON-mandatory characters and passes UTF-8 bytes
through untouched, so accented text survives intact (e.g. "...doivent etre
considerees comme approximatives et simplifiees..." renders with correct
accents, no replacement characters).

scene.zig's FeatureMeta serves these for native cells (via pickClass/pickJson)
in place of acronymByObjl + encodeS57Attrs; S-57 cells are unchanged.

Verified on the SHOM France cells: the pick report now carries the full S-101
attribute tree with correct UTF-8.
…TF-8)

Update the limitations note: the cursor-pick report now surfaces each native
feature's S-101 class and full attribute tree (nested complex attributes,
UTF-8 text). The remaining gap is StructureEquipment feature-to-feature
association resolution for related-feature grouping.
A native S-101 cell had no compilation scale, so bakeChartBytes (which read the
scale by parsing the file as S-57) fell back to cscl=0 -> the approach band
[z11,13] for EVERY native chart. A small-scale chart with a large coverage then
baked an enormous fine-zoom tile pyramid: an Arctic overview test cell
(101AA00AA1NPOL3, a 95 KB / 238-feature chart spanning 35deg x 9deg) took >200 s
and 1.35 trillion instructions, ~47% of it in memset (per-tile buffers) + gzip.

native.zig now reads the chart's scale from the DataCoverage feature's
optimumDisplayScale (falling back to maximum/minimum) and sets it as the cell's
cscl. bakeChartBytes and buildCellBackend parse native-aware (parseAnyCell) so
the band + coverage come from the real cell, not an S-57 misparse.

Result: NPOL3 200s+ -> 1.1s; 101US002EC03M 13.6s -> 0.4s; NPOLA/B 11s -> 0.1s;
harbor charts unchanged. Bands now match the S-101 navigational purpose
(US2->general, US3->coastal, US4->approach, US5->berthing), and equal the S-57
CSCL for the same chart (US2EC03M = 1:1,200,000 both ways).
UKHO S-101 datasets write nested parenthesized groups in the ISO 8211 field
format controls — e.g. (b11,b14,7A,A(8),3A,(b11)) and (b11,(3b24)) — where SHOM
cells use a flat list. parseSubfields treated a nested '(' as a subfield width
and fed the group body ('3b24') to the integer parser -> BadAsciiInt, which
aborted iso.parse for the WHOLE cell (every UKHO chart failed to load).

parseSubfields now recurses into balanced groups, and the width parse is tolerant
(a mis-aligned scan over binary notation like 'b24' yields width 0 instead of
throwing). The subfield defs are informational — s57/s101 decode records by their
fixed schema — so the only requirement is that a legal DDR parses.

Validated on 27 UKHO GB charts (all parse, 0 unresolved codes, 0 portrayal
errors) and the SHOM FR charts incl. their .001/.002 update chains; S-57 cells
and the iso8211 tests are unchanged.
S-57 ENC attribute text is ISO 8859-1 (Latin-1) at the standard lexical level —
a French SHOM chart's national name 'La Crabière Est' carries a lone 0xE8 for
'è'. parseATTF stored the ATVL bytes verbatim, so the tile / pick report /
rendered label carried invalid UTF-8 and a client showed the replacement char.

Transcode in parseATTF (covering ATTF and, via mergeNatf, the national NATF
names where accents live): a value already valid UTF-8 (ASCII or a UTF-8
producer) is kept as-is; otherwise each byte is taken as a Latin-1 codepoint and
UTF-8-encoded. Verified on FR471610 — the baked tile now carries 'Crabière' as
C3 A8; S-57 ASCII charts and native S-101 (already UTF-8) are unchanged.
@beetlebugorg beetlebugorg merged commit 9d21dc4 into main Jul 13, 2026
5 checks passed
@beetlebugorg beetlebugorg deleted the feat/s101-native branch July 13, 2026 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant